MIT OpenCourseWare 
http://ocw.mit.edu 



18.175 Theory of Probability 

Fall 2008 



For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms . 



Section 15 



Convergence of martingales. 
Fundamental Wald's identity. 



We finally get to our main result about the convergence of martingales and submartingalcs. 
Theorem 34 Let (X n ,B n )- oc<n<00 be a submartingale. 

1. The limit X-oc = lim„^_oo X n exists a.s. andEX^^ < +00. If B-^ = f] n B n then (X n , S„)_ 00 <„ <00 
is a submartingale. 

2. 7/sup„EX+ < 00, then X +oc , = lim^oo X n exists a.s. and EX^^ < 00. 

3. If (max(X„, a))_ 00< „ <00 is uniformly integrable for any a£l then (X n , S„)_ 00< „< +00 is a submartin- 
gale where X +00 is from part 2 and B +00 = o-(\J n B n ^j . 

Remark. The first statement says that a reverse submartingale always converges (because it is right-closed 
by definition). The second statement says that under integrability conditions a submartingale converges. 
The last statement means that under stronger uniform intergrability conditions the submartingale not only 
converges but the limit right-closes the submartingale. 
Proof. Let us note that X n converges as n — > 00 (00 — ±00) if and only if 

lim sup X n = lim inf X n . 

X n diverges if and only if the following event occurs 

jlimsup JT„ > lim inf X„| = |limsupX„ > b > a > lim inf X„|, 

a<b 

where the union is taken over all rational numbers a < b. In other words, F(X n diverges) > iff there exist 
rational a < b such that 

P(hmsupX„ > b > a > lim inf X r ^j > 0. 
If we recall that v(a, b, n) denotes the number of upcrossings of [o, b], this is equivalent to 

P ( lim is(a, 6, n) = 00 ) > 0. 

Proof of 1. Let us define a new sequence 

Y\ = X_ n ,Y 2 = X_ n+ i, . . . ,Y n = X_i. 
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By Doob's inequality, 

E^y (o, o, n) < = < oo. 

b - a b - a 

Since < v(a, b, n) | v(a, b), by monotone convergence theorem, 

Ev(a,b) = lim Ev(a,b,n) < oo. 

n^oo 

Therefore, F(v(a, b) = oo) = which means that the limit 

X_oo = lim X n 

n^ — oo 

exists. By Fatou's lemma and submartingalc property 

EXl^ < liminf EX+ n < EX+ X < oo. 



n — > — oo 



It remains to show that (X n , B n )- 00 < n<00 is a submartingalc. First of all, X_oo = lim n ^_oo X n is measurable 
on Z5 m for all m and, therefore, measurable on £>_oo = r\B m . Let us take a set A £ C\B n . We would like to 
show that for any m 

EX_ TO I A < EX rn l A . 
Since (X ni B n )-oo<n<v is a right-closed submartingale, by Lemma 29, part 2, 



(max(X„,a),B n ) 



-oo<n<0 

is uniformly intcgrablc for any aeR. Since 

max(X_oo,a) = lim max(X n ,a), 

n — y — oo 

by Lemma 30, 

Emax(X_oo, o)1a = lim Emax(X„, cl)Ia < Emax(X m , cl)\a 

n — > — oo 

for any m. Letting a — > — oo, by monotone convergence theorem, EX_ QO I J 4 < EX TO L4. 

Proo/ of 2. If i/(a, 6, n) is a number of upcrossings of [a, b] by Xi, . . . , X n then by Doob's inequality 

(b - a)Ev(a, b, n) < E(X n - a) + < \a\ + EX+ <K<oo. 

Therefore, Ev(a, b) < oo for v{a, b) — lim n ^oo v{a, 6, n) and, as above, X +QO — lim„^ +00 X n exists. Since all 
X n are measurable on B +oc> so is X +ao . Finally, by Fatou's lemma 

EX+ < liminf EX+ < oo. 

n^oo 

Notice that another condition, sup n E|X„| < oo, would similarly imply 

E|X +00 | < liminf E\X n \ < oo. 

n^oo 

Proof of 3. By 2, the limit X +oa exists. We want to show that for any set A £ B m 

EX m I^ < EX +oc Ia- 

Let us denote aVft = max(a, 6). Since (X n V a) is uniformly integrable and X n Vm X +ca V a, by Lemma 
30, 

E{X +oa V a)l A = lim E(X n V a)l A - 

n — >oo 

Since a function x — > x V a is convex and increasing, (JT„ V a) is also a submartingale and, thus, for n > m, 

E(X m V a)l A <E(X n V a)l A . 

Therefore, E(X m V o)Ia < E(X +oc V o)I^ and letting a — > — oo, by monotone convergence theorem, we get 
that EX m L < EX +00 I A . 

□ 

As a corollary we get the following. 
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Corollary 2 Martingale (X n ,B n ) is right-closable iff it is uniformly integrable. 

To prove this, apply case 3 above to (X n ) and (— X n ) which are both submartingales. 

Theorem 35 (Levy's convergence) Let (f2,B,P) be a probability space and X be a real-valued random vari- 
able on it. Given a sequence of a -algebras 

By C . . . C B n C . . . C B +QO C B 
where B +00 = c(lJi<n<cx> &n) > we have X n = E(X\B n ) — > E(X\B +oc ) a.s. 

Proof. (X n , B n )i< n<00 is a right-closable martingale since X n = E(X\B n ). Therefore, it is uniformly inte- 
grable and by previous theorem the limit X +QO = lim„^oo E(X\B n ) exists. It remains to show that 

X +00 =E(X\B +00 ). 

For a set A in algebra {JB n , A e B m for some m and by Lemma 30 

EX +oc l A = lim EX n l A = EX m l A = E(E(X\B m )l A ) = EXl A . 

n^oo 

By uniqueness in the Caratheodory extension theorem, EX +oc I A = EXI A for all A G &{V] n Br^j which 
means that X +QO = E(X\B +QO ). 

□ 

Exercise. (Kolmogorov's 0-1 law) Consider arbitrary random variables (Xj)j>i and consider cr-algebras 
B n = o~{Xi, . . . ,X n ) and B^ = cr((Xj)j> 1 ). A set A € B^ is a tail event if it is independent of B n for any 
n. If we consider conditional expectations E(L A \B n ) then using Levy's convergence theorem and the fact 
that E(L A \B n ) and L A are independent one can prove that P(A) = or 1. This gives a martingale proof of 
improved Kolmogorov's 0-1 law. 

Examples. 

1. (Strong law of large numbers) Let (X„)„>! be i.i.d. Let 

B- n = o-(S n , S n+ i, . . .), S n = Xi + ■ ■ ■ + X n . 

We showed before that 

Z- n = — =E{X 1 \B_ n ), 
n 

i.e. (Z^ n , B- n ) is a reverse martingale and, therefore, the limit Y = lim„^ +00 Z- n exists. Y is measurable on 
cr-algebra B-oo = f] n B- n . Since each event in B-oo is symmetric, i.e. invariant under permutations of X n 's, 
by Savage-Hewitt 0-1 law, the probability of each event is or 1, i.e. B-^ consists of and f2 up to sets 
of measure zero. Therefore, Y is a constant a.s. and since (^ n )-oo<n<-i is also a martingale, EY = EXi. 
Therefore, S n /n — > EX\ a.s. 

2. (Kolmogorov's law of large numbers) Consider a sequence (X n ) n >i such that 

E(X„ + i|Xi, . . . , X n ) = 0. 
We do not assume independence. If a sequence (b n ) is such that 

00 EX 2 

b n <b n+1 , lim b n = oo and — - < oo 
n^oo 6^ 
n=l " 

then S n /n — ► a.s. Indeed, F„ = ^2 k<n (Xk/bk) is a martingale and (F„) is uniformly integrable since 

1 1 00 FY 2 

E|F„|I(F„| > M) < -E|F n | 2 <-^y-0 

n=l ^ 
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as M — > oo. Therefore, the limit Y = limF„ exists and it is an easy exercise to show that S n /b n — ► 
(Kronccker's lemma). 

3. (Polya' urn scheme) Let us recall the Polya urn scheme from Section 5. Let us consider a sequence 

Y #(blue balls after n iterations) 
#(total after n iterations) 

Y n is a martingale because given that at step n the numbers of blue and red balls are b and r, the expected 
number of balls at step n + 1 will be 

b+r b+r+c b+r b+r+c b+r 

Since Y n is bounded, by martingale convergence theorem, the limit Y = lim„^oo Y n exists. What is the 
distribution of Y? Let us consider a sequence 



Xi 

and let S n = Y,i<„ x i- Clearly, 



1 blue at step i 
red at step i 



_ b + S n c ^ S n 
b + r + nc n 

as n — > oo and, therefore, S n /n — > Y. The sequence (X„) is exchangeable and by de Finetti's theorem in 
Section 5 we showed that 

P(S„ = k) = Q jf - x) n ~ k d^ (x). 
For any function u e C([0, 1]), 

"-(£) - 1>© (I) f **<i - *)-**('■ D w - f D <*>• 

where B n (x) is the Bernstein polynomial that approximates u(x) uniformly on [0, 1]. Therefore, 
which means that 

i.e. the limit Y has Beta distribution 

□ 

Optional stopping for martingales revisited. Let r be a stopping time. We would like to determine 
when EX T = EXi. As we saw above in the case of two stopping times, some kind of integrability assumptions 
are necessary. In this simpler case, the necessary conditions are clear from the proof. 

Lemma 31 We have 

EXx = lim EX t I(t < n) <^=+ lim EX n l(r > n) = 0. 

ro— »oo n— ► oo 

Proof. We can write, 

EX T l(T<n)= KX k l(r = k) = J2 ( EX k l(T > k) - EX k l(r > k + 1)) 

l<k<n l<k<n 

| since {r > k + 1} = {r < k} c e B k } = (EX k l(r > k) - EX k+1 l(r > k + l)j 



l<k<n 

EXi -EX„ +1 I(r > n + l). 
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Example. Given < p < 1, consider i.i.d. random variables (£,),>i such that 

P(& = i)=p, p(^ = -i) = i- p 

and consider a random walk X = 0, X„ + i = X n + £ n +i- Consider two integers a < — 1 and b > 1 and define 
a stopping time 

r = min{fc > 1, X n = aor 6}. 
If g = 1 — p then Y n = (q/p) Xn is a martingale since 

q\X n + l / gsX„-l /q\X„ 



E(Y n+1 \B n )=p p +5 

\p/ \p/ \p 



It is easy to show that P(t > n) — > as n — * oo and using previous lemma 

f 

<pJ ' ■ \p 

If p ^ 1/2 vie can solve this 



1 = EY = EY T = (?) P(X r = 6) + (?)°(1 - P(X T = 6)). 

(9/p) a - 1 



mT - h) ^ {q/pY-iq/pf 
If q = p = 1/2 then X n is a martingale and 

= b¥{X T = b)+ o(l - P(X T = 6)) => P(X T = 6) 



a — b 



Exercise. Compute Er. Hint: for p ^ 1/2, use that X n — nE£i is a martingale; for p = 1/2, use that AT^ — n 
is a martingale. 

□ 

Fundamental Wald's identity. Let (X n ) n >\ be an i.i.d. sequence of random variables and suppose that 
a Laplace transform ip{\) = Ee XXl is defined on some nontrivial interval (A_, A + ) containing 0. Since 



Ee AS " = (Ee XXl ) n = ip(\) n — is a martingale. 



Let r be a stopping time. If 
then by the above lemma we get 



e A5„ 



E— — — I(t > n) —f (15.0.1) 

E— — I(r < oo) = E — - = 1 

which is called the fundamental Wold 's identity. In some cases one can use this to obtain Laplace transform 
of a stopping time and, thus, its distribution. 

Example. Let X = 0,P(X j = ±1) = 1/2, S n = T, k <n X k- Given integer z > 1, let 

r = minjfc : Sk = —z or z). 

Since ip(X) = ch(A) > 1, 

e AS„ JA|z 

E— — I(t > n) < - . x . P(t > n) 
p(A) n v ~ ; ~ ch(A)™ v ~ ; 

and (15.0.1) is satisfied. Therefore, 

1 = E ^y7 = e Az Ech(A)- T I(5' r = z) + e - Az Ech(A)- T I(SV = -z) = & + * Ech(A)- T 
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by symmetry. Therefore, 



Ech(Ar T = — I— and Ee 7T = 1 

V ; ch(Az) chich- 1 {e-f)z) 



by a change of variables e 7 = 1 /chA. 

□ 

For more general stopping times the condition (15.0.1) might not be easy to check. We will now show another 
approach that is helpful to verify a fundamental Wald's identity. If P is the distribution of X^s, let V\ be a 
distribution with Radon-Nikodym derivative w.r.t. P given by 

dP x e Xx 



dP cp(\) ' 

This is, indeed, a density since 



Jr 



/r <fi(x) ¥>(A) 

We will think of (X n ) as defined on the product space (E°° , B°° , P°° ) . For example, a set 

{r = n}ea(X 1 ,...,X n )CB n 

is a Borel set on E™. We can write, 

e ASV °° e *s„ °° r e A(xi+-+x„) 
E — — I(r<oc) = > E— — I(r = n) = > / dP( Xl ) . . . dP(x n ) 

{r=n} 



]T / dP^i) ... rfP A (a;„) =P A (r<oo). 

n=l r J n 



{-r=n} 

This means that we can think of (X n ) as having a distribution F\ and to prove Wald's identity we need to 
show that Pa(t < oo) = 1. 

Example. ( Crossing a growing boundary) Suppose that we have a boundary given by a sequence (/(fc)) 
that changes with time and a stopping time (crossing time): 

r = min{fc : S k > /(&)}■ 

To verify (15.0.1), we need to show that ¥\(t < oo) = 1. Under the law F\, random variables Xi have 
expectation 



By the strong law of large numbers 



S n _ <ff{\) 



rt->oo n ¥?(A) 

and, therefore, if the growth of the crossing boundary satisfies 

/(») <p'W 

limsup^-^ < ^-v 
„^oo n tp(\) 

then, obviously, F\(t < oo) = 1. 



□ 
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